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ABSTRACT 


Ship  performance  characteristics,  such  as  max-sustained  speed,  acceleration,  and 
maneuverability  are  generally  pre-determined  as  a  platform  requirement  based  on 
precedents.  However,  these  pre-determined  performance  characteristics  have  far 
reaching  impacts  on  the  size,  logistics,  manning,  and  cost  of  the  ship  platform.  Instead  of 
designing  to  pre-defined  platform  performance  requirements,  ship  performance 
characteristics  should  be  determined  based  on  fulfilling  mission  objectives 

This  research  evaluates  the  viability  to  effectively  determine  if  the  ship 
characteristic  requirements  can  be  quantified  by  using  Naval  Sea  Systems  Command’s 
Naval  Battle  Engagement  Model  (NABEM) — an  agent-based  simulation  tool  developed 
by  Naval  Sea  Systems  Command.  In  particular,  we  study  two  tactical  situations  by 
varying  three  platform  characteristics — maximum  speed,  acceleration,  and  turning 
diameter — and  determine  how  these  platform  characteristics  affect  mission  performance. 
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made  within  the  time  available  to  ensure  the  models  are  free  of  computational  and  logic 
errors,  they  cannot  be  considered  fully  validated  models.  Any  application  of  these 
models  with  out  additional  validation  is  at  the  risk  of  the  user. 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


TABLE  OF  CONTENTS 


I.  INTRODUCTION . 1 

A.  BACKGROUND . 1 

B.  DISCUSSION . 2 

C.  OBJECTIVES . 2 

II.  METHODOLOGY . 3 

A.  PROBLEM  DEFINITION . 3 

B.  MEASURES  OF  PERFORMANCE . 3 

C.  MODELING  AND  SIMULATIONS . 5 

D.  DESIGN  OF  EXPERIMENTS . 6 

1.  Maximum  Speed . 6 

2.  Acceleration . 6 

3.  Maneuvering . 7 

E.  RESPONSE  SURFACE  MODEL . 8 

F.  ANALYSIS  OF  RESPONSE  SURFACE  MODEL . 9 

G.  ASSUMPTIONS  AND  CAVEATS . 9 

III.  TACTICAL  SITUATION  1 . 11 

A.  RED  FORCE . 12 

B.  BLUE  FORCE . 13 

C.  METRICS  FOR  TACTICAL  SITUATION  1 . 13 

D.  ANALYSIS  FOR  TACTICAL  SITUATION  1 . 13 

E.  RESULTS . 14 

1.  Max  Speed . 15 

2.  Acceleration . 16 

3.  Turn  Diameter . 16 

F.  CONCLUSIONS . 16 

IV.  TACTICAL  SITUATION  2 . 19 

A.  RED  FORCE . 20 

B.  BLUE  FORCE . 21 

C.  METRICS  FOR  TACTICAL  SITUATION  2 . 21 

D.  ANALYSIS  FOR  TACTICAL  SITUATION  2 . 21 

E.  RESULTS . 22 

1.  Max  Speed . 23 

2.  Acceleration . 24 

3.  Turn  Diameter . 24 

F.  CONCLUSIONS . 24 

V.  CONCLUSIONS . 27 

APPENDIX  A  RESPONSE  SURFACE  MODEL  FOR  TACTICAL  SITUATION  1 . 29 

A.  FIT  FOR  Ml.l . 29 

B.  FIT  FOR  M1.2 . 32 

C.  FIT  FOR  M1.3 . 35 


IX 


APPENDIX  B  RESPONSE  SURFACE  MODEL  FOR  TACTICAL  SITUATION  2 . 39 

A.  FIT  FOR  M2.1 . 39 

B.  FIT  FOR  M2.2 . 43 

C.  FIT  FOR  M2.3 . 46 

D.  FIT  FOR  M2.4 . 50 

LIST  OF  REFERENCES . 55 

INITIAL  DISTRIBUTION  LIST . 57 


x 


LIST  OF  FIGURES 


Figure  1.  TS1  General  Laydown . 11 

Figure  2.  TS1  NABEM  Model  Laydown . 12 

Figure  3.  Prediction  Profder  Results  for  TS1 . 15 

Figure  4.  TS2  General  Laydown . 19 

Figure  5.  TS2  NABEM  Model  Laydown . 20 

Figure  6.  Prediction  Profder  Results  for  TS2 . 23 

Figure  Al.  Summary  of  Model  Fit  For  Ml.l . 31 

F igure  A2 .  Summary  of  Model  Fit  For  M 1. 1  (continued) . 32 

Figure  A3.  Model  Fit  Error  For  Ml.l . 32 

Figure  A4.  Summary  of  Model  Fit  For  M1.2 . 34 

Figure  A5.  Summary  of  Model  Fit  For  M1.2  (continued) . 35 

Figure  A6.  Model  Fit  Error  For  M1.2 . 35 

Figure  A7.  Summary  of  Model  Fit  For  M1.3 . 37 

F igure  A8 .  Summary  of  Model  Fit  For  M 1 . 3  (continued) . 38 

Figure  A9.  Model  Fit  Error  For  Ml  .3 . 38 

Figure  B 1 .  Summary  of  Initial  Model  Fit  For  M2. 1 . 41 

Figure  B2.  Summary  of  Initial  Model  Fit  For  M2.1  (continued) . 41 

Figure  B3.  Pareto  Plot  For  M2.1  Initial  Fit . 41 

Figure  B4.  Summary  of  Model  Fit  For  M2. 1  HOT . 42 

Figure  B5.  Summary  of  Model  Fit  For  M2.1  HOT  (continued) . 43 

Figure  B6.  Model  Fit  Error  For  M2.1  HOT . 43 

Figure  B7.  Summary  of  Model  Fit  For  M2. 2 . 45 

Figure  B8.  Summary  of  Model  Fit  For  M2.2  (continued) . 46 

Figure  B9.  Model  Fit  Error  For  M2. 2 . 46 

Figure  BIO.  Summary  of  Initial  Model  Fit  For  M2. 3 . 48 

Figure  B 1 1 .  Summary  of  Initial  Model  Fit  For  M2. 3  (continued) . 48 

Figure  B12.  Pareto  Plot  For  M2.3Initial  Fit . 48 

Figure  B13.  Summary  of  Model  Fir  M2. 3  HOT . 49 

Figure  B14.  Summary  of  Model  Fit  For  M2. 3  HOT  (continued) . 50 

Figure  B15.  Model  Fit  Error  For  M2. 3  HOT . 50 

Figure  B16.  Summary  of  Model  Fit  For  M2.4 . 52 

Figure  B17.  Summary  of  Model  Fit  For  M2.4  (continued) . 53 

Figure  B18.  Model  Fit  Error  For  M2.4 . 53 


xi 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


LIST  OF  TABLES 


Table  1.  Notional  Measures  of  Performance . 4 

Table  2.  Speed  Range  of  Variations . 6 

Table  3.  Design  of  Experiments  Independent  Variables . 8 

Table  4.  Measures  of  Effectiveness  for  TS1 . 13 

Table  5.  Final  Results  for  TS1 . 14 

Table  6.  Measures  of  Effectiveness  for  TS2 . 21 

Table  7.  Final  Results  for  TS2 . 22 


xiii 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


xiv 


EXECUTIVE  SUMMARY 


Ship  performance  characteristics,  such  as  max-sustained  speed,  acceleration,  and 
maneuverability  are  generally  pre-determined  as  platform  requirements  based  on 
precedents.  However,  these  pre-determined  performance  characteristics  have  far 
reaching  impacts  on  the  size,  logistics,  manning,  and  cost  of  the  ship  platform.  Instead  of 
designing  to  pre-defined  platform  performance  constraints,  ship  performance 
characteristics  should  be  designated  to  fulfdl  mission  objectives. 

This  research  evaluates  the  viability  to  effectively  determine  if  the  ship 
characteristic  requirements  can  be  quantified  by  using  Naval  Sea  Systems  Command’s 
Naval  Battle  Engagement  Model  (NABEM) — an  agent-based  simulation  tool  developed 
by  Naval  Sea  Systems  Command.  In  particular,  we  study  two  tactical  situations  by 
varying  three  platform  characteristics — maximum  speed,  acceleration,  and  turning 
diameter — and  determine  how  these  platform  characteristics  affect  mission  performance. 

The  response  surfaces  generated  from  the  NABEM  simulations  produced 
insignificant  results.  Maximum  speed,  acceleration  and  turning  diameter  produced  only 
secondary  effects.  NABEM  is  primarily  an  engineering  simulation  model,  concerned 
more  with  the  detailed  mathematical  representation  of  individual  systems  or  components. 
It  provides  a  detailed  representation  of  sensor  and  weapon  systems,  not  platform 
characteristics.  For  this  reason,  it  appeared  that  sensor  and  weapons  have  a  stronger 
effect  on  the  results,  while  maximum  speed,  acceleration  and  turning  diameter  appeared 
as  secondary  effects  producing  little  or  no  effect. 

Implementation  of  the  methodology  allows  a  designer  to  assess  and  trade-off 
impacts  of  various  ship  characteristics  based  on  mission  effectiveness.  The  methodology 
provides  a  framework  where  feasible  and  economically  viable  alternatives  can  be 
identified  with  accuracy  along  with  their  effects  on  mission  effectiveness.  While  the 
methodology  is  capable  of  supporting  the  JCIDS  process,  NABEM  was  not  an  effective 
simulation  tool. 

Maximum  speed,  acceleration  and  turning  diameter  are  typically  tactical  decisions 
made  by  the  ship  operator  based  on  the  current  tactical  situation,  the  “human  in  the  loop”. 


xv 


If  the  platform  sensors  and  weapons  generate  the  tactical  decision  environment,  then  ship 
platform  performance — such  as  maximum  speed,  acceleration,  turning  diameter — may 
become  the  dominating  factors,  which  makes  it  possible  to  quantitatively  assess  their 
effects  on  mission  effectiveness.  Modeling  human  response  to  changing  tactical 
situations,  in  relation  to  platform  performance,  is  a  daunting  task.  Such  a  model  would 
greatly  enhance  future  ability  to  assess  ship  platform  characteristics  on  mission 
effectiveness  and  is  suggested  as  a  future  research  project. 


I. 


INTRODUCTION 


Ship  characteristics  are  generally  determined  based  on  historical  precedent. 
Characteristics  such  as  max  sustained1  speed,  signatures2,  acceleration,  and  maneuvering 
are  picked  based  on  what  was  done  previously,  unless  that  characteristic  was  proven  bad 
or  needed  improvement.  However,  such  decisions  can  have  far  reaching  impacts  to  the 
platform  size,  lifetime  logistics,  manning,  and  cost.  There  must  be  a  better  way  to 
quantitatively  determine  ship  characteristics  that  directly  relate  the  ships  ability  to 
perform  its  assigned  mission.  The  objective  of  this  work  is  to  determine  a  framework 
that  can  quantify  these  ship  characteristics  as  they  relate  to  operational  effectiveness. 

Operational  effectiveness  relates  operational  capability  to  operational 
performance  in  the  form  of  Measures  of  Performance  metrics  (MoPs).  However,  these 
MoPs  must  be  evaluated  within  a  specific  mission  and  operational  environment  to  be 
meaningful.  The  purpose  of  this  analysis  is  to  do  just  that,  within  the  proposed 
framework,  determine  the  effect  specific  ship  characteristics  have  on  the  overall 
operational  effectiveness  during  a  particular  mission. 

A.  BACKGROUND 

The  U.S.  Navy  has  shifted  its  emphasis  from  design  to  developing  broad  ship  and 
fleet  architectures  in  order  to  develop  design  requirements  to  meet  future  fleet 
architectures  as  well  as  deciding  on  the  merit  of  future  technologies  to  be  pursued.  The 
establishment  of  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  by 
the  Chairman  of  the  Joint  Chiefs  of  Staff  [13]  supports  this  shift  in  emphasis.  The  JCIDS 
process  requires  a  more  system  approach  to  determining  new  system  development.  The 
process  is  not  centered  on  the  platform  or  component  level  system,  but  on  the  integration 
and  impact  of  these  systems  on  joint/global  force  operations,  doctrine,  organization, 
training,  personnel,  material,  and  facilities.  Assessing  the  impacts  and  effectiveness  of 
these  complex  systems  becomes  an  increasingly  challenging  problem.  System  demands, 

1  Max  Sustain  speed  is  defined  as  the  maximum  speed  that  can  be  obtained  at  80%  of  the  ships  full 
power. 

2  Ship  signatures  include  radar  cross  section,  infrared  signatures,  and  acoustic  signature. 
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including  increased  performance,  lower  system  life  cycle  costs,  longer  operating 
capacities,  and  improved  productivity  and  efficiency,  must  be  balanced  against  limited 
resources,  scant  or  unknown  data,  the  identification  and  resolution  of  conflicts,  and 
resource  allocation  (people  and  cost). 

These  tradeoffs  point  to  the  need  for  an  integrated  and  systematic  framework  that 
can  assess  system  characteristics  as  it  affects  overall  system  effectiveness.  The  goal  of 
this  research  was  to  develop  an  analysis  framework  that  supports  the  JCIDS  process  by 
providing  a  methodology  that  provides  a  trade  off  environment  that  relates  platform 
performance  characteristics  to  operational  effectives  in  a  combat  environment.  This 
allows  a  sponsor  to  justify  (quantitatively)  operational,  material,  and  technology 
requirements  in  a  better/clearer  framework  in  which  to  develop  the  systems  level 
requirements  and  identify  future  technology  investments. 

B.  DISCUSSION 

In  order  to  provide  the  proper  framework  in  which  to  develop  and  evaluate 
platform  performance  characteristics  and  MoPs,  a  set  of  operational  and  mission 
requirements  are  generated.  These  requirements  are  in  the  form  of  Tactical  Situations, 
which  provide  the  operational  environment,  mission  characteristics,  goals,  tasks,  and 
threats.  These  tactical  situations  provide  the  framework  to  develop  simulations  to 
determine  the  MoPs,  therefore  relating  operational  effectiveness  to  system  performance. 

To  ensure  that  the  full  range  of  possible  platform  characteristics  are  addressed,  an 
operational  effectiveness  trade  space  is  developed  using  a  Design  of  Experiments  (DoE) 
methodology  coupled  with  a  Response  Surface  Model  (RSM). 

C.  OBJECTIVES 

The  objective  of  the  research  is  developing  a  trade  space  model  that  relates 
operational  effectiveness  to  platform  performance  characteristics.  In  addition,  provide 
traceable  linkages  between  measures  of  performance  associated  with  individual  platform 
characteristics,  and  measures  of  effectiveness  associated  with  the  required  mission. 
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II. 


METHODOLOGY 


The  operational  effectiveness  trade  space  was  developed  in  five  steps:  (1) 
defining  the  problem,  (2)  determining  operational  measures  of  performance  and  metrics, 
(3)  modeling  and  simulation  using  NAVSEA’s  Naval  Battle  Engagement  Model,  (4) 
design  of  experiments,  and  (5)  generating  a  response  surface  model.  This  approach  is 
similar  to  that  developed  by  the  Aerospace  Systems  design  Laboratory  at  Georgia 
Institute  of  Technology  and  is  know  as  the  Unified  Tradeoff  Environment 
[8][9][10][1 1][12][17]  and  assessed  the  impacts  of  system  requirements  on  the  system 
design  trade  space. 

A.  PROBLEM  DEFINITION 

In  order  to  formulate  the  problem,  it  is  assumed  that  there  was  a  need  for  a  new 
class  of  surface  combatant.  The  Initial  Capabilities  Document  outlines  the  general  ship 
capabilities,  but  left  several  ambiguous  requirements.  These  subjective  and  sometimes 
“fuzzy”  requirements  must,  or  should  be,  mapped  into  definitive  requirements.  The 
problem  therefore  is  to  develop  an  analysis  framework  to  best  determine  these  discrete 
design  requirements.  For  this  analysis  three  basic  ship  characteristics  are  evaluated, 
maximum  speed,  acceleration,  and  turning  diameter.  These  characteristics  generally  have 
significant  cost  and  ship  systems  impacts.  Determining  their  impact  on  mission 
effectiveness  would  help  define  their  relative  importance  to  the  overall  ship  system. 

B.  MEASURES  OF  PERFORMANCE 

In  order  to  make  logical  decisions  and  choices  for  the  three  ship  characteristics, 
criteria  to  measure  the  value  or  relative  importance  of  alternative  characteristics  are 
needed.  This  is  an  essential  part  of  an  operational  effectiveness  trade  space,  knowing 
what  metrics  are  to  be  used  to  determine  operational  success  or  failure  as  well  as  how  to 
quantify  these  metrics. 

For  this  thesis  the  Measures  of  Performance  (MoP)  correspond  to  individual 
mission  performances.  Typically  MoPs  are  quantitative  and  consist  of  a  range  of  values 


3 


about  a  desired  point.  These  values  are  performance  metrics  that  the  mission  targets,  by 
changing  system  characteristics,  so  as  to  finally  achieve  the  qualities  desired  for  the 
overall  mission.  MoPs  are  related  to  specific  missions  (i.e.,  tactical  situations)  and 
mission  tasks. 

The  set  of  MoPs  developed  for  each  tactical  situation  is  derived  form  the 
Universal  Joint  Task  List  (UJTL)  and  the  Navy  Tactical  Task  List  (NTTL)  [6]  [7].  The 
UJTL  and  the  NTTL  provides  relationships  between  missions,  operations,  and  tasks. 
These  relationships,  along  with  the  operational  analysis,  identify  the  operations  and  tasks 
that  must  be  performed  for  mission  success. 

The  mission  establishes  the  requirement  to  perform  tasks  and  provides  the  context 
for  each  task  performance  (including  the  conditions  under  which  a  task  would  be 
performed).  It  determines  where  and  when  a  task  must  be  performed  (one  or  more 
locations).  Finally,  it  determines  the  degree  to  which  a  task  must  be  performed  (implied 
in  the  concept  of  the  operations)  and  provides  a  way  to  understand  precisely  how  the 
performance  of  a  task  contributes  to  operational  success. 

This  thesis  studies  two  tactical  situations:  Tactical  Situation  1,  defense  of  a  major 
seaport,  and  Tactical  Situation  2,  defense  of  a  coastal  convoy.  Each  tactical  situation  is 
designed  to  stress  the  ship  characteristics  under  consideration.  Using  the  UJTL  and 
NTTL,  the  four  general  MoPs  were  determined  to  be  appropriate  for  the  analysis,  Table 
1. 


Units 

Measure  of  Performance 

M0.1 

Number 

Of  Blue  ships  damaged  by  enemy  attacks 

MO. 2 

Number 

Of  Blue  ship  sunk  by  enemy  attacks 

MO. 3 

Number 

Of  attacking  Red  ships  damaged. 

M0.4 

Number 

Of  attacking  Red  ships  destroyed. 

Table  1.  Notional  Measures  of  Performance 
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c. 


MODELING  AND  SIMULATIONS 


NAVSEA’s  Naval  Battle  Engagement  Model  (NABEM)  is  used  to  assess  the  ship 
characteristics  impacts  on  mission  performance,.  NABEM  is  the  primary  tool  for  the 
quantitative  portion  of  this  analysis.  NABEM  is  a  sophisticated  Monte  Carlo,  time-step, 
many-on-many  warfare  model  capable  of  accurately  simulating  all  tactical  interactions 
from  the  sea  surface  to  the  upper  limit  of  the  atmosphere  (air-to-air,  air-to-surface, 
surface-to-air,  and  surface-to-surface  engagements).  The  model  can  handle  any 
combination  of  air,  surface,  and  shore-based  platforms  and  associated  weapons,  and  also 
includes  neutral  surface  and  air  traffic  (merchantmen,  fishing  boats,  airliners,  etc.)  when 
desired. 

Created  in  the  1970s,  NABEM  has  been  in  continual  use  at  Naval  Surface  Weapons 
Center,  Carderock  Division  (NSWC-CD)  since  the  1980s.  It  has  undergone  continual 
development  and  upgrades  to  maintain  its  viability  and  extend  its  capabilities.  NABEM 
is  sensitive  to  a  ship’s  radar  cross  section  and  infrared,  visual,  and  emissions  signatures  in 
all  phases  of  an  engagement  for  both  aircraft  and  anti-ship  cruise  missiles  including 
detection,  targeting,  and  lock-on.  NABEM  is  also  sensitive  to  ship  passive  protection 
(vulnerability)  and  other  hull,  mechanical,  and  electrical  (HM&E)  technology  related 
issues. 

The  user  can  script  the  initial  behavior  of  ships  and  aircraft,  governing  their 
movement,  rules  of  engagement,  EMCON  status,  etc.  Movement  can  be  randomized  to 
any  extent  desired,  allowing  NABEM  to  vary  the  scenario  geometry.  NABEM  has  a 
limited  Artificial  Intelligence  capability,  in  which  both  blue  and  red  units  act  only  on  the 
information  they  possess:  that  is,  platforms  in  NABEM  never  operate  with  a  “gods-eye” 
view,  as  often  occurs  in  other  models.  Platforms  in  NABEM  can  operate  only  on  their 
situational  awareness  at  the  platform  and  force  level,  so  that  tactical  and  targeting 
decisions  are  based  on  the  information  held  by  each  platform  at  each  particular  moment. 

The  model  allows  for  the  representation  of  the  events  occurring  in  an  engagement 
in  a  Naval  Tactical  Data  System  (NTDS)-like  graphics  display.  This  capability  is  a 
valuable  tool  for  evaluating  and  validating  initial  scenario  geometry  and  tactics,  tracing 
key  events,  troubleshooting,  and  demonstrating  the  model  in  briefings. 
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D.  DESIGN  OF  EXPERIMENTS 


The  Design  of  Experiments  (DoE)  is  a  statistical  driven  process  that  allows  for  the 
maximization  of  experimental  data.  With  a  minimum  number  of  trials  (in  this  case 
simulation  runs),  a  large  number  of  system  parameters  can  be  quantitatively  examined  to 
understand  the  effect  each  parameter  has  on  the  overall  system. 

The  analysis  concerns  the  effect  maximum  speed,  acceleration,  and  turning 
diameters  have  on  mission  effectiveness.  These  three  parameters  are  the  three 
independent  input  variables  used  for  the  DoE. 

1.  Maximum  Speed 

Maximum  speed  is  varied  as  an  independent  variable;  measured  in  knots.  This 
variable  will  determine  if  ship  speed  contributes  to  the  overall  outcome  of  the  tactical 
situation.  The  ability  of  the  ship  to  provide  power  to  maintain  max  speed  will  ultimately 
be  compared  to  the  outcomes  of  the  tactical  simulation  to  produce  a  response  surface. 

The  ranges  of  speeds  for  the  DoE  are  listed  in  Table  2.  Max  speeds  were  chosen 
to  bracket  current  and  potential  surface  combatant  max  speeds. 


Low 

High 

Max  Speed  (kts) 

25.0 

45.0 

Acceleration  (kts/sec) 

0.1 

0.5 

Turning  Diameter  (ft) 

1000 

2500 

Table  2.  Speed  Range  of  Variations 


2.  Acceleration 

A  range  of  acceleration  is  varied  as  independent  variable;  measured  in  knots  per 
second.  This  variable  determines  if  the  ships  acceleration  contributes  to  the  overall 
outcome  of  the  tactical  situation.  The  ability  of  the  ship  to  provide  power  to  accelerate 
will  ultimately  be  compared  to  the  outcomes  of  the  tactical  simulation  to  produce  a 
response  surface. 
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The  ranges  of  accelerations  for  the  DoE  are  listed  in  Table  2.  Accelerations  are 
chosen  to  bracket  current  and  potential  surface  combatant  accelerations. 

3.  Maneuvering 

Turning  diameter  varies  the  ships  maneuvering  characteristics,  measured  in  feet. 
This  variable  will  determine  if  maneuvering  affects  the  overall  outcome  of  the  tactical 
situation.  During  mission  execution,  the  ship  must  make  maneuvers  to  avoid  or  engage 
threats. 

The  ranges  of  turning  diameters  are  listed  in  the  Table  2.  Turning  diameters  high 
and  low  values  are  based  on  current  and  potential  surface  combatant  turning  diameters. 

The  independent  variable  ranges  for  maximum  speed,  acceleration  and  tuning 
diameter  represent  technically  feasible  solutions  for  the  ship  platforms  being  considered. 
When  the  region  of  interest  is  the  same  as  the  region  of  feasibility,  the  best  DoE  model  to 
chose  is  a  design  cube  model. 

There  are  several  different  design  cube  models  to  choose  from,  each  having  their 
own  pros  and  cons.  For  this  analysis  a  face-center  central  composite  design  was  chosen. 
The  face-center  central  composite  design  is  an  efficient  design  that  is  ideal  for  sequential 
experimentation  and  allows  a  reasonable  amount  of  information  for  testing  of  model  fit 
while  not  involving  an  unusually  large  number  of  design  points. 

Each  DoE,  one  for  Tactical  Situation  1  and  one  for  Tactical  Situation  2 
systematically  varies  these  three  independent  variables,  Table  3. 
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Design 

Points 

Max  Speed 
(kt) 

Acceleration 

(kt/sec) 

Turn  Dia 
(ft) 

1 

25 

0.5 

1000 

2 

45 

0.5 

1000 

3 

45 

0.3 

1750 

4 

45 

0.1 

1000 

5 

25 

0.5 

2500 

6 

45 

0.1 

2500 

7 

35 

0.1 

1750 

8 

25 

0.3 

1750 

9 

25 

0.1 

1000 

10 

35 

0.3 

1750 

11 

35 

0.5 

1750 

12 

45 

0.5 

2500 

13 

35 

0.3 

1000 

14 

35 

0.3 

2500 

15 

25 

0.1 

2500 

Table  3.  Design  of  Experiments  Independent  Variables 


Each  design  point  is  simulated  in  NABEM  for  5000  independent  runs.  The 
resulting  statistics  from  each  design  point  provides  the  operational/mission  effects  or 
MoPs  related  to  the  variations  in  ship  characteristics 

E.  RESPONSE  SURFACE  MODEL 

The  RSM  is  a  multi-variable  regression  technique  that  models  the  response  of  a 
complex  system  using  a  simple  equation.  The  response  surface  is  modeled  using  a 
second-order  quadratic  equation  thus  giving  a  model  of  the  relationships  between  the 
independent  (input)  variables  and  the  responses  obtained  for  the  simulation  model,  in  this 
case  NABEM.  The  response  surface  is  modeled  using  a  second-order  quadratic  equation 
and  is  expressed  as 


*=*.+z*,*,+zv,2+z  iv* 

i= 1  i= 1  i= 1  j=i+ 1 

Where:  by  are  the  coefficients  for  the  cross-product  terms. 

bi  are  the  regression  coefficients  for  the  first-degree  Xi  and  Xj  are  the  design  variables. 

terms.  bn  are  the  coefficients  for  the  pure  quadratic  terms. 
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The  RSM  is  generated  using  data  provided  by  the  DoE  and  the  NAMBEM 
simulations.  This  data  is  fed  into  SAS’s  JMP  software  to  generate  the  RSM.  JMP 
(Version  5.1)  is  a  statistical  analysis  software  tool  that  links  statistics  with  graphics  to 
interactively  explore,  understand,  and  visualize  data.  The  software  is  designed  to  uncover 
relationships  and  outliers  in  the  data.  JMP  provides  statistical  tools  as  well  as  Design  of 
Experiments  and  Statistical  Quality  Control.  JMP’s  built-in  capabilities  are  used  to 
develop  the  DoE,  the  RSMs,  and  JMP’s  interactive  graphic  tools  to  explore  the  design 
space.  In  particular  JMP’s  profder,  contour  profder,  and  surface  graphic  displays  are 
used. 

F.  ANALYSIS  OF  RESPONSE  SURFACE  MODEL 

The  response  surfaces,  of  RSM,  form  the  basis  or  framework  of  the  operational 
effectiveness  trade  space  model.  Through  visualization  tools  built  into  the  analysis  tool 
(JMP),  the  trade  space  can  be  analyzed.  The  results  are  displayed  through  a  series  of 
visualization  tools,  prediction  and  contour  profilers,  and  response  surface  plots.  The 
profiler  plots  displays  prediction  traces  (predicted  responses  as  one  variable  is  changed 
while  holding  the  others  constant)  for  each  variable.  The  response  surface  plots  provide  a 
quick  visual  of  how  the  response  functions  are  behaving. 

The  prediction  profiler  isolates  the  impact  of  every  factor  for  every  response.  The 
contour  profiler  illustrates  interaction  of  one  or  several  of  the  variables  have  on  one 
another.  The  prediction  and  contour  profilers  are  interactive  plots,  therefore,  very  difficult 
to  show  in  a  written  report.  Therefore,  only  the  general  ship  characteristics  effects  on 
mission  performance  are  described  for  each  tactical  situation. 

G.  ASSUMPTIONS  AND  CAVEATS 

A  conceptual  surface  combatant  is  developed  specifically  for  this  study  and  does 
not  relate  to  any  program  of  record  or  any  potential  U.S.  Navy  program.  This  conceptual 
ship  is  designated  as  the  Small  Surface  Combatant  (SSC)  for  this  analysis.  Mission 
systems  for  the  SSC  were  modeled  to  be  consistent  with  current  or  projected  systems  and 
are  constant  and  consistent  throughout  the  different  tactical  situations.  To  maintain  the 
unclassified  natural  of  this  thesis,  sensor  and  weapon  performance  data  is  generated  from 
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open  source  literature.  It  should  be  noted  that  the  intent  of  the  analysis  is  not  to  evaluate 
mission  systems  performance  or  to  compare  individual  mission  system  capabilities, 
performance,  or  effectiveness.  The  focus  of  the  analysis  is  to  determine  the  effect  of 
specific  ship  characteristics  on  the  ship’s  overall  mission  performance. 

The  Red  Force  combat  systems  and  platforms  are  chosen  to  be  representative  of 
likely  current  and  future  threats;  again  data  is  obtained  from  open  source  resources. 

The  tactical  situations  reflect  missions  that  stress  the  ship  characteristics  being 
evaluated.  They  are  designed  to  resemble  possible  real  world  situations,  but  the  locations 
of  these  scenarios  are  kept  generic  to  maintain  the  unclassified  nature  of  this  analysis. 

NABEM  is  a  good  combat  simulation  model;  however,  like  most  simulation 
models  it  has  its  limitations.  One  of  these  limitations  is  that  the  simulation  has  to  be  well 
scripted,  not  allowing  for  a  human  in  the  loop  decisions  as  to  course,  speed,  weapons  to 
use,  etc.  that  can  determine  the  outcome  of  a  simulation. 

Blue  force  platforms  does  not  include  helicopter  capability  and  is  not  included  in 
the  NABEM  simulation. 
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III. 


TACTICAL  SITUATION  1 


Tactical  Situation  1  (TS1)  models  the  defense  of  a  major  seaport  against  enemy 
forces  trying  to  gain  entry  into  the  port.  Blue  force’s  mission  is  to  prevent  enemy  forces 
(Red  Force)  from  gaining  access  to  the  port.  The  port  of  San  Francisco  is  chosen  for  this 
tactical  situation. 

Blue  forces  are  deployed  at  the  mouth  and  within  San  Francisco  harbor  in 
preparation  of  Red  hostilities.  Three  blue  small  surface  combatants  are  assigned  a  patrol 
area  each.  Two  patrol  the  approaches  to  the  harbor,  while  the  third  patrols  the  inner 
harbor  area,  as  shown  in  Figure  1.  Figure  2  shows  the  TS1  scenario  as  modeled  in 
NABEM. 


Red  Attack  -  West 


Red  Attack  -  South 


Figure  1.  TS1  General  Laydown 
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Red  Force-West 


Red  Force-South  ^ 

Figure  2.  TS1  NAB  EM  Model  Laydown 


Blue  Force-Patrol  Areas 


Red  Force-North 

X 


The  red  force  comprises  three  separate  squadrons,  each  squadron  approaching  the 
harbor  from  different  directions,  as  shown  in  Figure  1.  The  north  and  the  south 
squadrons  had  three  high  speed  attack  craft,  while  the  western  squadron  had  six.  The 
mission  goal  for  Red  Forces  goal  is  to  penetrate  the  harbor  defenses  and  disrupt  harbor 
operations. 

A.  RED  FORCE 

The  red  force  is  chosen  to  be  representative  of  the  likely  threat  for  this  tactical 
situation.  Each  red  force  platform,  designated  as  Red  PTG,  was  modeled  as  a  small  fast 
attack  craft  with  operating  characteristics  similar  to  the  Peoples  Republic  of  China  Type 
083  Fast  Attack  craft  and  the  Iranian  Navy’s  Bohammer  fast  attack  craft.  Red  PTG  has  a 
maximum  speed  of  35  knots,  20.0  nautical  mile  range  navigational  radar,  and  an  ESM 
detection  system. 

Each  Red  PTG  carries  six  short-range  missiles.  Each  missile  has  a  maximum 
operating  range  of  approximately  2.0  nautical  miles  and  has  similar  operating 
characteristics  to  hand-launched  anti-armor  missiles  and  rocket  propelled  grenades. 
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B.  BLUE  FORCE 


Blue  forces  consist  of  three  small  surface  combatants,  designated  as  Blue  SSC. 
Each  Blue  SSC  platform  is  modeled  as  a  corvette  size  combatant  similar  to  the  German 
Type  143  and  148  Fast  Attack  Craft  and  the  Swedish  Goteborg  Class  Corvettes.  Blue 
SSC  forms  the  basis  for  the  design  of  experiments  in  which  max  speed,  accelerations  and 
turning  diameter  are  varied. 

Each  Blue  SSC  has  a  medium  caliber  gun  (similar  to  the  Mk  57  Naval  Gun),  24 
short  range  missiles,  a  point  defense  gun  (similar  to  the  Mk  15  CIWS),  an  ESM  detection 
and  decoy  system,  and  a  200  nautical  mile  surface  search  radar  (similar  to  a  SPY- IF 
radar). 

C.  METRICS  FOR  TACTICAL  SITUATION  1 

The  MoPs  chosen  for  TS1  reflect  measurable  mission  characteristics  that  are 
influenced  by  the  three  ship  characteristics  being  investigated  (Table  4).  The  metrics  are 
designed  to  evaluate  the  effectiveness  of  the  Blue  Forces  against  the  threat  in  each 
particular  scenario. 


Units 

Measure  of  Effectiveness 

Ml.l 

Number 

Red  PTGs  Survive  and  Enter  Harbor 

M1.2 

Number 

Blue  SSCs  Sunk 

Ml. 3 

Number 

Blue  SSCs  Damaged 

Table  4.  M 

[easures  of  Effectiveness  for  TS1 

D.  ANALYSIS  FOR  TACTICAL  SITUATION  1 

The  DoE  is  run  in  NABEM  according  to  the  DoE  matrix  in  Table  3.  There  are  15 
design  points;  each  design  point  is  simulated  5000  times,  each  with  the  initial  simulation 
parameters  randomly  chosen.  The  simulations  results  are  listed  in  Table  5.  Response 
surface  models  for  each  MoE  are  generated  from  the  results  listed  in  Table  5.  Each 
response  is  checked  for  RSquare  values,  Adjusted  RSquared  values,  F  statistics,  and 
Model  Fit  Error  to  ensure  the  accuracy  of  the  model  fit.  If  any  of  the  checks  fails,  it  is  an 
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indication  that  the  basic  second-order  model  is  not  appropriate  for  the  response.  In  these 
cases  two  options  are  available  to  provide  a  better  fit,  adding  higher  terms  or  adding  more 
design  points.  For  all  the  MoEs  only  one  of  the  checks  fail,  Model  Fit  Error.  Higher 
order  terms  are  tried,  but  fail  more  than  one  of  the  checks.  Generating  more  design 
points  was  considered,  but  due  to  time  and  resources  was  not  possible.  For  this  thesis 
the  response  surface  models  generated  from  the  design  points  in  Table  6  are  the  best  fit 
possible.  For  a  more  thorough  examination  the  TSl’s  response  model  are  referred  to 
Appendix  A. 

E.  RESULTS 


■ 

Max  Spd 

Accel 

Turn  Dia 

PTGs  Enter 

Harbor 

SSCs  Sunk 

SSCs  Damaged 

(kt) 

(kt/sec) 

(«) 

Mean 

%  of  Total 

Mean| 

%  of  Total 

Mean 

%  of  Total 

1 

25 

MEM 

0.979 

(8.2) 

0.330 

(11.0) 

1.141 

(38.0) 

2 

45 

wm 

1.149 

(9.6) 

0.413 

(13.8) 

1.410 

(47.0) 

3 

45 

0.3 

1750 

1.224 

(10.2) 

0.392 

(13.1) 

1.404 

(46.8) 

4 

45 

0.1 

1000 

1.202 

-10 

0.404 

(13.5) 

1.421 

(47.4) 

5 

25 

0.5 

2500 

1.021 

(8.5) 

0.317 

(10.6) 

1.124 

(37.5) 

6 

45 

0.1 

2500 

1.161 

(9.7) 

0.382 

(12.7) 

1.424 

(47.5) 

7 

35 

0.1 

1750 

0.883 

(7.4) 

0.377 

(12.6) 

1.299 

(43.3) 

8 

25 

0.3 

1750 

0.998 

(8.3) 

0.320 

(10.7) 

1.157 

(38.6) 

9 

25 

0.1 

1000 

0.857 

(7.1) 

0.313 

(10.4) 

1.170 

(39.0) 

10 

35 

0.3 

1750 

0.926 

(7.7) 

0.365 

(12.2) 

1.304 

(43.5) 

11 

35 

0.5 

1750 

0.938 

(7.8) 

0.334 

(11.1) 

1.137 

(37.9) 

12 

45 

0.5 

2500 

1.217 

(10.1) 

0.393 

(13.1) 

1.401 

(46.7) 

13 

35 

0.3 

1000 

0.924 

(7.7) 

0.377 

(12.6) 

1.281 

(42.7) 

14 

35 

0.3 

2500 

0.936 

(7.8) 

0.370 

(12.3) 

1.305 

(43.5) 

15 

25 

0.1 

2500 

0.926 

(7.7) 

0.319 

(10.6) 

1.113 

(37.1) 

Table  5.  Final  Results  for  TS1 


From  the  response  surface  model  for  each  MoE  a  prediction  profiler  plot  is 
generated  which  shows  the  interrelationships  between  all  the  parameters,  as  shown  in 
Figure  3.  The  prediction  profiler  serves  as  the  principal  tool  for  evaluating  the  individual 
responses.  The  following  sections  discuss  the  key  findings  for  each  of  the  ship 
characteristics  investigated. 
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Figure  3.  Prediction  Profiler  Results  for  TS1 


1.  Max  Speed 

Referring  to  the  prediction  profiler  in  Figure  3;  as  maximum  speed  increases  all 
three  MoEs  increase.  Meaning,  as  Blue  SSC’s  speed  increases  the  number  of  Red  PTGs 
successfully  entering  the  harbor  also  increases,  and  so  does  the  number  of  SSCs  damaged 
or  sunk.  This  behavior  seems  to  contradict  general  conventional  rules  of  thumb,  but  as 
speed  increases,  Blue  SSCs  are  exposed  to  more  Red  PTGs.  So  why  then  does  the 
number  of  PTGs  surviving  go  up  and  not  down  with  similar  weapons  and  sensors. 
Simple  Red  PTGs  outnumber  the  Blue  SSCs  by  4  to  1.  Therefore,  there  is  a  higher 
probability  that  Blue  will  incur  damage  or  sink.  With  fewer  Blue  ships  available  the 
probability  that  more  PTGs  survive  to  complete  their  mission  (enter  the  harbor)  increases. 
Another  factor  to  consider,  as  modeled  in  the  simulation,  Blue  SSCs  have  a  limited 
amount  of  ready  service  ammunition.  As  Red  PTGs  attack,  Blue  deplete  their  ready 
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service  ammunition.  The  simulation  model  reloads  ammunition  over  time  and  this  time 
delay  is  a  contributing  factor  to  the  increased  number  of  Blue  SSCs  damaged  or  sunk. 

While  maximum  speed  has  an  effect  on  the  outcome  of  the  tactical  situation,  the 
effect  is  insignificant. 

2.  Acceleration 

Referring  to  the  prediction  profiler  in  Figure  3,  as  acceleration  increases,  the 
number  of  Blue  SSCs  sunk  or  damaged  (Ml. 2  and  Ml. 3)  tend  to  decrease  and  the 
numbers  of  Red  PTGs  survive  to  enter  the  harbor  (Ml.l)  increase.  This  behavior  seems 
to  contradict  general  conventional  rules  of  thumb.  However,  tactical  situation  1  iss  set  in 
a  very  confined  operating  space  (San  Francisco  Harbor)  negating  possible  advantages 
acceleration  might  provide  in  an  open  ocean  operational  area.  In  addition,  the  increase  or 
decrease  in  acceleration  effects  on  the  MoE  are  very  small  compared  to  the  other  ship 
characteristics.  Combined  the  overall  effect  from  acceleration  is  negligible. 

3.  Turn  Diameter 

Referring  to  the  prediction  profiler  in  Figure  3,  as  the  turning  diameter  increases, 
the  number  of  Blue  SSCs  sunk  or  damaged  (Ml. 2  and  Ml. 3)  tend  to  decrease  and  the 
number  of  Red  PTGs  survives  to  enter  the  harbor  (Ml.l)  increase.  This  behavior  also 
seems  to  contradict  general  conventional  rules  of  thumb.  However,  the  initial  turning 
diameter  of  1000  ft  produces  a  high  number  of  Blue  SSCs  damaged  or  sunk.  This  is 
consistent  with  the  results  when  maximum  speed  is  increased.  A  highly  maneuverable 
ship  in  confined  waters  increases  the  probability  of  exposure  to  enemy  weapons, 
therefore  increasing  the  probability  of  damage  or  sinking  Blue  SSCs.  Factor  in  the  Red 
PTGs  four  to  one  superiority  in  numbers  the  results  can  be  understood.  These  effects  are 
very  small  and  are  insignificant. 

F.  CONCLUSIONS 

The  final  response  surface  models  developed  for  tactical  situation  1  produced 
tangible  models  that  link  ship  characteristic  to  mission  effectiveness;  however,  the  effects 
are  small  and  insignificant.  NABEM  is  primarily  an  engineering  simulation  model, 
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concerned  more  with  the  detailed  mathematical  representation  of  individual  systems  or 
components.  NABEM  provides  a  detailed  representation  of  sensor  and  weapon  systems, 
not  platform  characteristics.  For  this  reason  sensor  and  weapons  effects  dominate  the 
results.  Maximum  speed,  acceleration  and  turning  diameter  appear  as  secondary  effects 
producing  little  or  no  impact  to  the  overall  outcome  of  the  tactical  situation. 
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IV 


TACTICAL  SITUATION  2 


Tactical  Situation  2  (TS2)  models  the  defense  of  a  blue  convoy  transiting  through 
a  wide  strait  off  the  coast  of  Blue  territory.  Red  forces  attack  the  convoy  from  two 
different  directions  and  at  different  times.  For  this  tactical  scenario  the  Straits  of  Florida 
is  modeled,  with  the  Blue  convoy,  comprised  of  ten  merchant  ships,  transiting  north 
through  the  straits,  as  shown  in  Figure  4.  Figure  5  shows  the  TS2  scenario  as  modeled  in 
NABEM. 


Red  Attack-East 


Red  Attack-South 


Blue  Convoy  Route 


Figure  4.  TS2  General  Laydown 
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Figure  5.  TS2  NABEM  Model  Laydown 


The  red  force  attacks  the  blue  convoy  from  two  different  directions.  First,  a 
single  Red  ship  attacks  the  convoy  from  the  rear,  hoping  to  draw  the  two  Blue 
combatants  away  from  the  convoy.  The  second  attack  of  four  Red  ships  comes  from  the 
east  and  aims  at  the  lead  ship  of  the  convoy,  as  shown  in  Figure  4. 

A.  RED  FORCE 

The  red  force  is  chosen  to  be  representative  of  the  likely  threat  for  this  tactical 
situation.  Each  red  force  platform,  designated  as  Red  PTG,  was  modeled  as  a  small  fast 
attack  craft  with  operating  characteristics  to  Peoples  Republic  of  China  Type  083  Fast 
Attack  craft  with  a  maximum  speed  of  35  knots,  20  nautical  mile  range  navigational 
radar,  and  an  ESM  detection  system. 

Each  Red  PTG  carries  six  short-range  missiles.  Each  missile  has  a  maximum 
operating  range  of  approximately  6.0  nautical  miles  and  similar  operating  characteristics 
to  the  Hellfire  missile. 
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B.  BLUE  FORCE 


Blue  forces  consist  of  three  small  surface  combatants,  designated  as  Blue  SSC. 
Each  Blue  SSC  platform  is  modeled  as  a  corvette  size  combatant  similar  to  the  Swedish 
Goteborg  Class  Corvettes.  Blue  SSC  forms  the  basis  for  the  design  of  experiments  in 
which  max  speed,  accelerations  and  turning  diameter  are  varied. 

Each  Blue  SSC  has  a  medium  caliber  gun  (similar  to  the  Mk  57  Naval  Gun),  24 
short  range  missiles,  a  point  defense  gun  (similar  to  the  Mk  15  CIWS),  an  ESM  detection 
and  decoy  system,  and  a  200  nautical  mile  surface  search  radar  (similar  to  a  SPY- IF 
radar). 

Blue  force  convoy  ships  are  modeled  as  typical  coastal  commercial  tankers  and 
were  designated  as  Blue  Tankers. 

C.  METRICS  FOR  TACTICAL  SITUATION  2 

The  measures  of  performance  for  TS2  reflect  measurable  mission  metrics  that  are 
influenced  by  the  three  ship  characteristics  being  investigated,  as  shown  in  Table  6.  The 
metrics  are  designed  to  evaluate  the  effectiveness  of  the  Blue  Forces  against  the  threat  in 
each  particular  scenario. 


Units 

Measure  of  Effectiveness 

M2.1 

Number 

Of  Blue  SSCs  damaged  by  enemy  attacks 

M2. 2 

Number 

Of  Blue  SSCs  sunk  by  enemy  attacks 

M2. 3 

Number 

Of  Blue  Tankers  sunk  by  enemy  attacks 

M2.4 

Number 

Of  attacking  Red  PTG  sunk 

Table  6.  1 

Measures  of  Effectiveness  for  TS2 

D.  ANALYSIS  FOR  TACTICAL  SITUATION  2 

The  DoE  is  run  in  NAB  EM  according  to  the  DoE  matrix  in  Table  3.  There  are  15 
design  points;  each  design  point  was  simulated  5000  times,  each  with  the  initial 
simulation  parameters  randomly  chosen.  The  results  from  the  simulations  are  listed  in 
Table  7.  A  response  surface  model  is  generated  from  the  results  in  Table  7.  Each 
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response  is  checked  for  RSquare  values,  Adjusted  RSquared  values,  F  statistics,  and 
Model  Fit  Error  to  ensure  the  accuracy  of  the  model  fit.  If  any  of  the  checks  fail,  it  is  an 
indication  that  the  basic  second-order  model  is  not  appropriate  for  the  response.  In  these 
cases  two  options  were  available  to  provide  a  better  fit,  adding  higher  terms  or  adding 
more  design  points.  For  all  the  MoEs  at  least  one  of  the  checks  failed.  Higher  order 
terms  were  tried  for  all  the  design  points.  Adding  [Turn  Diaemeter  x  Acceleration]  to 
M2.1  and  [Max  Speed  x  Acceleration]  to  M2. 3  makes  it  possible  to  pass  three  of  the  four 
checks,  but  all  the  response  surface  model  still  fail  one  check,  Model  Fit  Error. 
Generating  more  design  points  was  considered,  but  due  to  time  and  resources  was  not 
possible.  For  this  thesis  the  response  surface  models  from  the  design  points  in  Table  7 
are  the  best  fit  possible.  For  a  more  thorough  examination  the  TS2’s  model  fit,  see 
Appendix  B. 

E.  RESULTS 


Max  Spd 

Accel 

Turn  Dia 

Blue  SSCs 
Damaged 

SSCs  Sunk 

Tankers  Sunk 

Red  PTG  Sunk 

(kp 

(kt/sec) 

(ft) 

Mean 

%  of  Total 

Mean 

%  of  Total 

Mean 

%  of  Total 

Mean 

%  of  Total 

1 

25 

0.5 

1000 

0.397 

(13.3) 

0.181 

(6.0) 

0.338 

(3.4) 

2.374 

(59.4) 

2 

45 

0.5 

1000 

0.432 

(14.4) 

0.228 

(7.6) 

0.205 

(2.1) 

3.449 

(86.2) 

3 

45 

0.3 

1750 

0.461 

(15.4 

0.221 

(7.4) 

0.203 

(2.0) 

3.431 

(85.8) 

4 

45 

0.1 

1000 

0.351 

(11.7) 

0.190 

(6.3) 

0.205 

(2.1) 

3.495 

(87.4) 

5 

25 

0.5 

2500 

0.582 

(19.4) 

0.238 

(7.9) 

0.337 

(3.4) 

2.376 

(59.4) 

6 

45 

0.1 

2500 

0.384 

(12.8) 

0.177 

(5.9) 

0.206 

(2.1) 

3.480 

(87.0) 

7 

35 

0.1 

1750 

0.344 

(11.5) 

0.165 

(5.5) 

0.230 

(2.3) 

2.937 

(73.4) 

8 

25 

0.3 

1750 

0.545 

(18.1) 

0.219 

(7.3) 

0.338 

(3.4) 

2.379 

(59.5) 

9 

25 

0.1 

1000 

0.402 

(13.4) 

0.184 

(6.1) 

0.345 

(3.5) 

2.382 

(59.5) 

10 

35 

0.3 

1750 

0.458 

(15.3) 

0.201 

(6.7) 

0.233 

(2.3) 

2.886 

(72.2) 

11 

35 

0.5 

1750 

0.577 

(19.2) 

0.208 

(6.9) 

0.342 

(3.4) 

2.347 

(58.7) 

12 

45 

0.5 

2500 

0.564 

(18.8) 

0.263 

(8.8) 

0.214 

(2.1) 

3.383 

(84.6) 

13 

35 

0.3 

1000 

0.408 

(13.6) 

0.177 

(5.9) 

0.245 

(2.4) 

2.912 

(72.8) 

14 

35 

0.3 

2500 

0.483 

(16.1) 

0.216 

(7.2) 

0.223 

(2.2) 

2.845 

(71.1) 

15 

25 

0.1 

2500 

0.520 

(17.3) 

0.213 

(7.1) 

0.356 

(3.6) 

2.330 

(58.2) 

Table  7.  Final  Results  for  TS2 


From  the  response  surface  model  for  each  MoE  a  prediction  profiler  plot  is 
generated  which  shows  the  interrelationships  between  all  the  parameters,  Figure  6.  The 
prediction  profiler  serves  as  the  principal  tool  for  evaluating  the  individual  responses. 


22 


The  following  sections  discuss  the  key  findings  for  each  of  the  ship  characteristics 
investigated. 


Prediction  Profiler 
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Max  Spd  Accel  Turn  Dia 

Figure  6.  Prediction  Profiler  Results  for  TS2 

1.  Max  Speed 

Referring  to  the  prediction  profiler  in  Figure  6,  maximum  speed  has  an  overall 
positive  effect.  As  maximum  speed  increases  the  number  of  Blue  Tankers  sunk  (M2. 3) 
decrease  and  the  number  of  PTGs  sunk  (M2.4)  increase.  This  makes  sense,  as  the 
number  of  PTGs  goes  down,  so  should  the  number  of  Tankers  sunk  because  the  Tankers 
are  exposed  to  fewer  PTGs  and  thus  lowers  the  probability  of  the  tankers  getting  sunk. 

However,  as  maximum  speed  increases,  the  number  of  Blue  SSCs  damaged  or 
sunk  (M2.1  and  M2.2)  increase.  The  explanation  for  this  is,  as  speed  increases  Blue 
SSCs  are  exposed  to  more  potential  combat  with  Red  PTGs,  increasing  the  number  of 
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Red  PTGs  sunk,  but  which  also  increases  Blue  SSCs  exposure  to  Red  weapons, 
increasing  the  probability  of  Blue  SSCs  getting  damaged  or  sunk. 

While  maximum  speed  has  an  effect  on  the  outcome  of  the  tactical  situation,  the 
effect  is  insignificant. 

2.  Acceleration 

Referring  to  the  prediction  profiler  in  Figure  6,  acceleration  has  an  overall 
negative  effect  on  Blue  forces.  As  acceleration  increases  the  number  of  Blue  SSCs  and 
Tankers  sunk  or  damaged  increase.  But  the  number  of  Red  PTGs  sunk  trends  to 
decrease.  This  appeared  to  be  a  contradiction,  but  as  more  PTGs  survive,  more  Blue 
ships  are  exposed  to  enemy  fire,  thus  have  a  higher  probability  of  getting  damaged  or 
sunk.  Therefore,  increasing  the  ship’s  acceleration  would  increase  the  number  of  PTGs 
sunk,  but  in  this  instance  the  change  is  so  small  that  it  can  be  ignored. 

3.  Turn  Diameter 

Referring  to  the  prediction  profiler  in  Figure  6,  turning  diameter  has  a  positive 
effect  on  the  number  Blue  SSCs  damaged  or  sunk  and  the  number  of  Blue  Tankers  sunk 
(M2.1,  M2.2,  and  M2. 3).  Also,  as  turning  diameter  gets  tighter  the  number  of  Red  PTGs 
sunk  goes  up  initially,  the  plot  for  turning  diameter  for  M2. 4  as  a  concave  structure 
indicating  for  a  turning  diameter  of  approximately  1750  ft  the  numbers  of  PTGs  sunk  are 
at  a  minimum.  The  model  suggests  that  turning  diameter  was  beneficial  to  the  outcome 
of  Blue  forces,  but  these  benefits  are  small.  The  small  changes  in  the  number  of  Red 
PTGs  sunk  are  not  significant. 

F.  CONCLUSIONS 

The  final  response  surface  models  developed  for  tactical  situation  2  produced  tangible 
models  that  link  ship  characteristic  to  mission  effectiveness;  however  the  effects  are 
small  and  insignificant.  NABEM  is  primarily  an  engineering  simulation  model, 
concerned  more  with  the  detailed  mathematical  representation  of  individual  systems  or 
components.  NABEM  provides  a  detailed  representation  of  sensor  and  weapon  systems, 
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not  platform  characteristics.  For  this  reason  sensor  and  weapons  effects  dominated  the 
results.  Maximum  speed,  acceleration  and  turning  diameter  appear  as  secondary  effects 
producing  little  or  no  impact  to  the  overall  outcome  of  the  tactical  situation. 
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y. 


CONCLUSIONS 


Response  surfaces  generated  from  the  NABEM  simulations  produce  insignificant 
results.  Maximum  speed,  acceleration  and  turning  diameter  produce  only  secondary 
effects,  while  sensor  and  weapons  effects  dominate  the  results.  NABEM  is  primarily  an 
engineering  simulation  model,  concerned  more  with  the  detailed  mathematical 
representation  of  individual  systems  or  components.  It  provides  a  detailed  representation 
of  sensor  and  weapon  systems,  not  platform  characteristics.  For  this  reason  sensor  and 
weapons  effects  dominates  the  results,  while  maximum  speed,  acceleration  and  turning 
diameter  appeared  as  secondary  effects  producing  little  or  no  effect. 

Implementation  of  the  methodology  allows  a  designer  to  assess  and  trade-off 
impacts  of  various  ship  characteristics  based  on  mission  effectiveness.  The  methodology 
provides  a  framework  where  feasible  and  economically  viable  alternatives  can  be 
identified  with  accuracy  along  with  their  effects  on  mission  effectiveness.  While  the 
methodology  is  capable  of  supporting  the  JCIDS  process,  NABEM  is  not  an  effective 
simulation  tool  to  use  in  the  process. 

Maximum  speed,  acceleration  and  turning  diameter  are  typically  tactical  decisions 
made  by  the  ship  operator  based  on  the  current  tactical  situation,  the  “human  in  the  loop”. 
If  the  platform  sensors  and  weapons  generate  the  tactical  decision  environment,  then  ship 
platform  performance — such  as  maximum  speed,  acceleration,  turning  diameter — may 
become  the  dominating  factors,  which  makes  it  possible  to  quantitatively  assess  their 
effects  on  mission  effectiveness.  Modeling  human  response  to  changing  tactical 
situations,  in  relation  to  platform  performance,  is  a  daunting  task.  Such  a  model  would 
greatly  enhance  future  ability  to  assess  ship  platform  characteristics  on  mission 
effectiveness  and  is  suggested  as  a  future  research  project. 
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APPENDIX  A  RESPONSE  SURFACE  MODEL  FOR 
TACTICAL  SITUATION  1 


The  following  sections  provide  the  general  procedures  to  develop  the  response 
surface  model  fits  for  each  measure  of  effectiveness  for  tactical  situation  1 .  All  analysis 
work  is  preformed  using  SAS’s  statistical  modeling  tool  JMP  (Version  5.1). 

A.  FIT  FOR  Ml.l 

The  results  for  the  NABEM  tactical  situation  1  simulation  for  Ml. 2  MoE  arre 
feed  into  JMP.  The  fit  model  option  is  used  to  do  an  initial  fit  of  the  data  using  the 
surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 
RSquared  and  RSquared  Adjusted  values  are  0.988  and  0.967  respectively,  Figure  A1 
Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 
Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0003,  which  is  well  below  the  0.05  for  a 
desired  95%  confidence  level.  The  basic  model  statistics  look  good. 

Second,  a  review  of  two  key  plots  is  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  Al,  all  the  data  points  are  falling  within  the  95%  confidence  lines  (the 
dashed  red  line),  again  indicating  a  good  fit  to  the  data.  Two,  the  Residual  by  Predicted 
Plot,  Figure  A2,  shows  a  fairly  evenly  shattering  of  the  points.  Some  areas  in  the  plot 
have  holes,  but  that  can  be  attributed  to  the  small  data  set  being  analyzed,  otherwise 
everything  indicates  a  good  model  fit. 

All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure  Al 
is  a  good  model  fit  to  the  data.  One  additional  test  is  needed  to  verify  the  accuracy  of  the 
model,  a  model  fit  error  analysis. 

The  model  fit  error  analysis  checks  how  well  the  model  fits  the  data  points  from 
the  design  of  experiments.  The  model  fit  error,  measured  as  the  model  percent  error,  is 
determined  for  each  data  point  and  the  resulting  distribution  is  evaluated  against  two 
error  distribution  criteria;  a  mean  of  approximately  0.0  and  a  standard  deviation  of  less 
than  1 .0  are  desired.  The  model  percent  error  is  computed  for  each  using  the  following 
equation: 
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{[(Predicted  Value)  -  (M  1.1  Actual)]  /(Ml.l  Actual)}  x  100 


Figure  A3  shows  the  resulting  distribution  for  the  model  percent  error.  The 
distribution  does  not  look  like  a  normal  distribution,  which  is  what  would  be  expected. 
In  addition,  the  mean  is  0.017,  which  is  close  to  0.0,  but  the  standard  deviation  is  1.39, 
which  is  higher  than  desired.  Based  on  the  model  fit  error  analysis,  the  model  for  Ml.l 
does  not  perform  as  well  as  would  be  expected  from  the  initial  analysis  of  the  fit.  This  is 
due  most  likely  to  the  small  data  set  used  for  the  analysis.  An  additional  set  of  data 
points  should  be  added  to  the  analysis,  to  help  better  define  the  model.  However,  due  to 
time  and  limited  resources,  that  is  not  possible,  and  the  fit  will  have  to  suffice  with  the 
caveat  that  the  final  analysis  can  only  provide  general  trends,  not  qualitative  answers. 
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Actual  by  Predicted  Plot 


RSquare  0.988398 

RSquareAdj  0.967514 

Root  Mean  Square  Error  0.023519 

Mean  of  Ftesponse  1.022707 

Observatbns  (or  SumWgts)  15 


Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

9 

0.23562197 

0.026180 

47.3289 

Error 

5 

0.00276578 

0.000553 

Prob  >  F 

C.  Total 

14 

0.23838775 

0.0003 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

2.2018056 

0.18119 

12.15 

<.0001 

Max  Spd 

-0.100403 

0.010549 

-9.52 

0.0002 

Accel 

0.9544208 

0.283497 

3.37 

0.0200 

Turn  Dia 

0.0001225 

0.000101 

1.21 

0.2797 

Max  Spd*Max  Spd 

0.0016942 

0.000147 

11.55 

<.0001 

Max  Spd*Accel 

-0.013388 

0.004158 

-3.22 

0.0235 

AcceFAccel 

-0.780833 

0.366671 

-2.13 

0.0865 

Max  SpdTurn  Da 

-0.000001 

0.000001 

-1.27 

0.2614 

AcceFTum  Dia 

0.0000692 

0.000055 

1.25 

0.2674 

Turn  Dia*Turn  Dia 

-2.113e-8 

2.607e-8 

-0.81 

0.4546 

Figure  Al.  Summary  of  Model  Fit  For  Ml.l 
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Effect  Tests 


Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

Max  Spd 

1 

1 

0.05010935 

90.5882 

0.0002 

Accel 

1 

1 

0.00626944 

1 1 .3340 

0.0200 

Turn  Dia 

1 

1 

0.00081246 

1.4688 

0.2797 

Max  SpcHVbx  Spd 

1 

1 

0.07380516 

133.4257 

<.0001 

Max  Spd*Accel 

1 

1 

0.00573521 

10.3682 

0.0235 

Accel*Accel 

1 

1 

0.00250848 

4.5349 

0.0865 

Max  Spd*Turn  Da 

1 

1 

0.00088621 

1.6021 

0.2614 

AccerTum  Dia 

1 

1 

0.00086112 

1.5568 

0.2674 

Turn  Dia*Turn  Dia 

1 

1 

0.00036312 

0.6565 

0.4546 

Residual  by  Predicted  Plot 


0.02-1 — 
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Quantiles 


100.0% 

maximum 

2.453 

99.5% 

2.453 

97.5% 

2.453 

90.0% 

1.969 

75.0% 

quartile 

1.351 

50.0% 

median 

-0.329 

25.0% 

quartile 

-1.299 

10.0% 

-1.709 

2.5% 

-1.875 

0.5% 

-1.875 

0.0% 

minimum 

-1.875 

Moments 

Mean 

0.0173557 

Std  Dev 

1.3884707 

Std  Err  Mean 

0.3585016 

upper  95%  Mean 

0.7862652 

lower  95%  Mean 

-0.751554 

N 

15 

Figure  A2.  Summary  of  Model  Fit  For  Ml .  1 
(continued) 


Figure  A3.  Model  Fit  Error  For 
Ml.l 


B.  FIT  FOR  M1.2 

The  results  for  the  NAB  EM  tactical  situation  1  simulation  for  the  Ml. 2  MoE,  are 
feed  into  JMP.  The  fit  model  option  is  used  to  do  an  initial  fit  of  the  data  using  the 
surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 
RSquared  and  RSquared  Adjusted  values  are  0.931  and  0.808  respectively,  Figure  A4 
Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 
Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0191,  which  is  well  below  the  0.05  for  a 
desired  95%  confidence  level.  The  basic  model  statistics  look  good. 
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Second,  a  review  of  two  key  plots  was  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  A4,  all  the  data  points,  with  the  exception  of  two  points,  fall  within  the  95% 
confidence  lines  (the  dashed  red  line),  indicating  a  good  fit  to  the  data.  Two,  the 
Residual  by  Predicted  Plot,  Figure  A5,  shows  a  fairly  evenly  shattering  of  the  points. 
Some  areas  in  the  plot  have  holes,  but  that  can  be  attributed  to  the  small  data  set  being 
analyzed,  otherwise  everything  indicates  a  good  model  fit. 

All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure  A4 
is  a  good  model  fit  to  the  data. 

For  the  model  fit  error  analysis,  Figure  A6  shows  the  resulting  distribution  for  the 
model  percent  error.  The  distribution  does  not  look  like  a  normal  distribution,  which  is 
what  would  be  expected.  In  addition,  the  mean  is  0.061,  which  is  close  to  0.0,  but  the 
standard  deviation  is  2.28,  which  was  much  higher  than  desired.  Based  on  the  model  fit 
error  analysis,  the  model  for  Ml. 2  does  not  perform  as  well  as  would  be  expected  from 
the  initial  analysis  of  the  fit.  This  is  due  most  likely  to  the  small  data  set  used  for  the 
analysis.  An  additional  set  of  data  points  should  be  added  to  the  analysis,  to  help  better 
define  the  model.  However,  due  to  time  and  limited  resources,  that  is  not  possible,  and 
the  fit  will  have  to  suffice  with  the  caveat  that  the  final  analysis  can  only  provide  general 
trends,  not  qualitative  answers. 


33 


Actual  by  Predicted  Plot 


RSquare  0.93161 

RSquare  Adj  0.808507 

Root  Mean  Square  Error  0.015241 

Mean  of  Ftesponse  0.360433 

Observatbns  (or  SumWgts)  15 


Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

9 

0.01582198 

0.001758 

7.5677 

Error 

5 

0.00116151 

0.000232 

Prob  >  F 

C.  Total 

14 

0.01698349 

0.0191 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Profc»|t| 

Intercept 

0.1516173 

0.117419 

1.29 

0.2531 

Max  Spd 

0.0098716 

0.006836 

1.44 

0.2083 

Accel 

0.1288625 

0.183718 

0.70 

0.5143 

Turn  Dia 

-0.000046 

0.000065 

-0.71 

0.5112 

Max  Spd*Max  Spd 

-0.000073 

0.000095 

-0.77 

0.4778 

Max  Spd*Accel 

0.0003125 

0.002694 

0.12 

0.9122 

AcceFAccel 

-0.195972 

0.237618 

-0.82 

0.4471 

Max  SpdTurn  Da 

-5.767e-7 

7.185e-7 

-0.80 

0.4586 

AcceFTum  Dia 

-0.000015 

0.000036 

-0.41 

0.7000 

Turn  Dia*Turn  Dia 

1.8153e-8 

1.69e-8 

1.07 

0.3318 

Figure  A4.  Summary  of  Model  Fit  For  M 1 .2 
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Error  Ml. 2 


Effect  Tests 


Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

Max  Spd 

1 

1 

0.00048440 

2.0852 

0.2083 

Accel 

1 

1 

0.00011429 

0.4920 

0.5143 

Turn  Dia 

1 

1 

0.00011608 

0.4997 

0.5112 

Max  Spd*IVbx  Spd 

1 

1 

0.00013661 

0.5881 

0.4778 

Max  Spd*Accel 

1 

1 

0.00000313 

0.0135 

0.9122 

Accel*Accel 

1 

1 

0.00015801 

0.6802 

0.4471 

Max  Spd*Turn  Da 

1 

1 

0.00014964 

0.6442 

0.4586 

AccePTum  Dia 

1 

1 

0.00003872 

0.1667 

0.7000 

Turn  Dia*Turn  Dia 

1 

1 

0.00026811 

1.1542 

0.3318 

Residual  by  Predicted  Plot 


0.03-1 - 

0.02- 

1  0.01- 
"D 

c/)  m  .  ' 

£  0.00- . ■ . . . ;• . , . 

Csl 

^  -0.01  - 
-0.02- 

-0.03-1 - 1 - 1 - 1 - 1 - 

.325  .350  .375  .400  .425 

Ml  .2  Predicted 


Figure  A5.  Summary  of  Model  Fit  For  Ml. 2 
(continued) 


Quantiles 


Quantiles 

100.0%  maximum  6.310 

99.5%  6.310 

97.5%  6.310 

90.0%  4.029 

75.0%  quartile  1.190 

50.0%  median  -0.421 

25.0%  quartile  -0.969 

10.0%  -3.546 

2.5%  -5.371 

0.5%  -5.371 

0.0%  minimum  -5.371 

Moments 


Mean 

0.0616479 

Std  Dev 

2.5828597 

Std  Err  Mean 

0.6668915 

upper  95%  Mean 

1.4919879 

lower  95%  Mean 

-1.368692 

N 

15 

Figure  A6.  Model  Fit  Error 
For  Ml. 2 


C.  FIT  FOR  M1.3 

The  results  for  the  NABEM  tactical  situation  1  simulation  for  the  Ml. 3  MoE,  are 
feed  into  JMP.  The  fit  model  option  is  used  to  do  an  initial  fit  of  the  data  using  the 
surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 

RSquared  and  RSquared  Adjusted  values  are  0.943  and  0.840  respectively,  Figure  A7 

Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 

Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0125,  which  is  well  below  the  0.05  for  a 

desired  95%  confidence  level.  The  basic  model  statistics  look  good. 
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Second,  a  review  of  two  key  plots  is  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  A7,  all  the  data  points,  with  the  exception  of  one  point,  fall  within  the  95% 
confidence  lines  (the  dashed  red  line),  indicating  a  good  fit  to  the  data.  Two,  the 
Residual  by  Predicted  Plot,  Figure  A8,  shows  a  fairly  evenly  shattering  of  the  points. 
Some  areas  in  the  plot  have  holes,  but  that  can  be  attributed  to  the  small  data  set  being 
analyzed,  otherwise  everything  indicates  a  good  model  fit. 

All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure  A7 
is  a  good  model  fit  to  the  data. 

For  the  model  fit  error  analysis,  Figure  A9  shows  the  resulting  distribution  for  the 
model  percent  error.  The  distribution  does  not  look  like  a  normal  distribution,  which  is 
what  would  be  expected.  In  addition,  the  mean  is  0.054,  which  is  close  to  0.0,  but  the 
standard  deviation  is  2.38,  which  was  much  higher  than  desired.  Based  on  the  model  fit 
error  analysis,  the  model  for  Ml. 3  does  not  perform  as  well  as  would  be  expected  from 
the  initial  analysis  of  the  fit.  This  is  due  most  likely  to  the  small  data  set  used  for  the 
analysis.  An  additional  set  of  data  points  should  be  added  to  the  analysis,  to  help  better 
define  the  model.  However,  due  to  time  and  limited  resources,  that  is  not  possible,  and 
the  fit  will  have  to  suffice  with  the  caveat  that  the  final  analysis  can  only  provide  general 
trends,  not  qualitative  answers. 
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Actual  by  Predicted  Plot 


RSquare  0.942996 

RSquare  Adj  0.840388 

Root  Mean  Square  Error  0.048598 

Mean  of  Ftesponse  1.272847 

Observatbns  (or  SumWgts)  15 


Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

9 

0.19534854 

0.021705 

9.1903 

Error 

5 

0.01180886 

0.002362 

Prob  >  F 

C.  Total 

14 

0.20715740 

0.0125 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

1.1899542 

0.374395 

3.18 

0.0246 

Max  Spd 

-0.000472 

0.021797 

-0.02 

0.9836 

Accel 

0.5637875 

0.585794 

0.96 

0.3800 

Turn  Dia 

-0.000239 

0.000209 

-1.15 

0.3040 

Max  Spd*Max  Spd 

0.0001762 

0.000303 

0.58 

0.5863 

Max  Spd*Accel 

-0.000963 

0.008591 

-0.11 

0.9152 

AcceFAccel 

-1.132083 

0.757656 

-1.49 

0.1954 

Max  SpdTurn  Da 

0.0000011 

0.000002 

0.49 

0.6447 

AcceFTum  Dia 

0.000024 

0.000115 

0.21 

0.8423 

Turn  Dia*Turn  Dia 

5.291 9e-8 

5.388e-8 

0.98 

0.3711 

Figure  A7.  Summary  of  Model  Fit  For  Ml  .3 
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Error  Ml. 3 


Effect  Tests 


Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Proto  F 

Max  Spd 

1 

1 

0.00000111 

0.0005 

0.9836 

Accel 

1 

1 

0.00218766 

0.9263 

0.3800 

Turn  Dia 

1 

1 

0.00309731 

1.3114 

0.3040 

Max  Spd*Max  Spd 

1 

1 

0.00079803 

0.3379 

0.5863 

Max  Spd*Accel 

1 

1 

0.00002965 

0.0126 

0.9152 

AccePAccel 

1 

1 

0.00527292 

2.2326 

0.1954 

Max  Spd*Turn  Da 

1 

1 

0.00056785 

0.2404 

0.6447 

AccePTurn  Dia 

1 

1 

0.00010368 

0.0439 

0.8423 

Turn  Dia*Turn  Dia 

1 

1 

0.00227843 

0.9647 

0.3711 

Residual  by  Predicted  Plot 


0.075  - 

0.050- 

1  0.025- 
;o 

|  0.000- . 

CO 

5-0.025- 

-0.050- 


1.10  1.15  1.20  1.25  1.30  1.35  1.40  1.45 
Ml  .3  Predicted 


Quantiles 


100.0% 

maximum 

5.948 

99.5% 

5.948 

97.5% 

5.948 

90.0% 

3.814 

75.0% 

quartile 

1.434 

50.0% 

median 

-0.320 

25.0% 

quartile 

-1.421 

10.0% 

-3.088 

2.5% 

-3.943 

0.5% 

-3.943 

0.0% 

minimum 

-3.943 

Moments 

Mean 

0.0535017 

Std  Dev 

2.3775419 

Std  Err  Mean 

0.6138787 

upper  95%  Mean 

1.3701406 

lower  95%  Mean 

-1.263137 

N 

15 

Figure  A8.  Summary  of  Model  Fit  For  Ml  .3 
(continued) 


Figure  A9.  Model  Fit  Error  For 
Ml. 3 
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APPENDIX  B  RESPONSE  SURFACE  MODEL  FOR 
TACTICAL  SITUATION  2 


The  following  sections  provide  the  general  procedures  to  develop  the  response 
surface  model  fits  for  each  measure  of  effectiveness  for  tactical  situation  2.  All  analysis 
work  is  preformed  using  SAS’s  statistical  modeling  tool  JMP  (Version  5.1). 

A.  FIT  FOR  M2.1 

The  results  for  the  NABEM  tactical  situation  1  simulation  are  feed  into  JMP.  The 
fit  model  option  is  used  to  do  an  initial  fit  of  the  data  using  the  surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 
RSquared  and  RSquared  Adjusted  values  are  0.879  and  0.662  respectively,  Figure  B1 
Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 
Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0682,  which  is  above  the  0.05  for  a 
desired  95%  confidence  level.  The  basic  model  statistics  do  not  look  great  and  deserve 
some  more  attention. 

Second,  a  review  of  two  key  plots  is  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  Bl,  all  the  data  points  are  falling  within  the  95%  confidence  lines  (the 
dashed  red  line),  indicating  a  good  fit  to  the  data.  Two,  the  Residual  by  Predicted  Plot, 
Figure  B2,  shows  a  fairly  evenly  shattering  of  the  points.  Some  areas  in  the  plot  have 
holes,  but  that  can  be  attributed  to  the  small  data  set  being  analyzed,  otherwise  the  plots 
indicate  good  model  fit. 

To  fix  the  fit  statistics,  higher  order  terms  (HOT)  are  added  to  the  model.  The 
two  terms  that  have  the  most  influence  from  the  initial  fit  are  (Turn  Dia)  and  (Accel) 
(refer  to  the  Pareto  Plot  in  Figure  B3).  Running  combinations  of  higher  order  terms  using 
Turn  Dia  and  Accel,  the  best  results  are  achieved  by  adding  the  term  [Turn  Dia  *  Turn 
Dia  *  Accel].  The  new  RSquared  and  RSquared  Adjusted  values  are  0.981  and  0.933 
respectively,  and  the  Analysis  of  Variance  the  F  statistic  is  0.0053,  Figure  B4. 

The  new  Actual  by  Predicted  Plot  (Figure  B4)  and  Residual  by  Predicted  Plot 
(Figure  B5)  show  the  same  similar  characteristics  as  the  initial  fit 
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All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure  B4 
is  a  good  model  fit  to  the  data.  One  additional  test  is  needed  to  verify  the  accuracy  of  the 
model,  a  model  fit  error  analysis. 

The  model  fit  error  analysis  checks  how  well  the  model  fits  the  data  points  from 
the  design  of  experiments.  The  model  fit  error,  measured  as  the  model  percent  error,  is 
determined  for  each  data  point  and  the  resulting  distribution  is  evaluated  against  two 
error  distribution  criteria;  a  mean  of  approximately  0.0  and  a  standard  deviation  of  less 
than  1 .0  are  desired.  The  model  percent  error  is  computed  for  each  using  the  following 
equation: 

{[(Predicted  Value)  -  (Ml.l  Actual)]  /  (Ml.l  Actual)}  x  100 

Figure  6  shows  the  resulting  distribution  for  the  model  percent  error.  The 
distribution  does  not  look  like  a  normal  distribution,  which  is  what  would  be  expected. 
In  addition,  the  mean  is  0.034,  which  is  close  to  0.0,  but  the  standard  deviation  is  2.46, 
which  is  higher  than  desired.  Based  on  the  model  fit  error  analysis,  the  model  for  M2.1 
does  not  perform  as  well  as  would  be  expected  from  the  initial  analysis  of  the  fit.  This  is 
due  most  likely  to  the  small  data  set  used  for  the  analysis.  An  additional  set  of  data 
points  should  be  added  to  the  analysis,  to  help  better  define  the  model.  However,  due  to 
time  and  limited  resources,  that  is  not  possible,  and  the  fit  will  have  to  suffice  with  the 
caveat  that  the  final  analysis  can  only  provide  general  trends,  not  qualitative  answers. 
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Actual  by  Predicted  Plot 


Summary  of  Fit 


RSquare 

0.879522 

RSquare  Adj 

0.662663 

Root  Mean  Square  Error 

0.047242 

Mean  of  Response 

0.46052 

Observatbns  (or  SumWgts) 

15 

Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

9 

0.08146320 

0.009051 

4.0557 

Error 

5 

0.01115888 

0.002232 

Prob  >  F 

C.  Total 

14 

0.09262208 

0.0687 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.562202 

0.363946 

1.54 

0.1831 

Max  Spd 

-0.021751 

0.021189 

-1.03 

0.3517 

Accel 

-0.195535 

0.569444 

-0.34 

0.7453 

Turn  Dia 

0.0002962 

0.000203 

1.46 

0.2042 

Max  Spd*Max  Spd 

0.0002767 

0.000295 

0.94 

0.3907 

Max  Spd*Accel 

0.0128187 

0.008351 

1.53 

0.1854 

AcceFAccel 

-0.364444 

0.736509 

-0.49 

0.6417 

Max  Spd*Turn  Da 

-0.000002 

0.000002 

-1.03 

0.3513 

AcceFTum  Dia 

0.0001381 

0.000111 

1.24 

0.2700 

Turn  Dia*Turn  Dia 

-5.294e-8 

5.237e-8 

-1.01 

0.3585 

Effect  Tests 


Source 

Nparm 

DF  Sum  of  Squares 

F  Ratio 

Prob  >  F 

Max  Spd 

1 

1  0.00235164 

1.0537 

0.3517 

Accel 

1 

1  0.00026315 

0.1179 

0.7453 

Turn  Dia 

1 

1  0.00475420 

2.1302 

0.2042 

Max  Spd*Max  Spd 

1 

1  0.00196908 

0.8823 

0.3907 

Max  Spd*Accel 

1 

1  0.00525825 

2.3561 

0.1854 

AcceFAccel 

1 

1  0.00054646 

0.2449 

0.6417 

Max  Spd*Turn  Da 

1 

1  0.00235641 

1.0558 

0.3513 

AcceFTum  Dia 

1 

1  0.00343206 

1.5378 

0.2700 

Turn  Dia*Turn  Dia 

1 

1  0.00228013 

1.0217 

0.3585 

Residual  by  Predicted  Plot 


0.075  - 

■ 

0.050- 

|  0.025-  ... 

|  0.000- . * . i . 

I -0.025-  ■  * 

-0.050  - 

■ 

-0.075  -I - 1 - 1 - 1 - 1 - 1 - 1 - 

.35  .40  .45  .50  .55  .60  .65 

M2.1  Predicted 


Figure  B 1 .  Summary  of  Initial  Model  Fit  For  M2. 1 


Figure  B2.  Summary  of  Initial  Model  Fit  For  M2. 1  (continued) 


Pareto  Plot  of  Transformed  Estimates 


Term 

Accel 

Turn  Dia 

Max  Spd 

Max  Spd*Accel 

AcceTTum  Dia 

MaxSpd*Turn  Dia 

Turn  Dia*Tum  Dia 

Accel  *Accel 

Max  Spd*Max  Spd 


Orthog  Estimate 
0.0450869 
0.0442541 
-0.0207308 
0.0187230 
0.0151263 
-0.0125337 
-0.0123292 
-0.0099742 
0.0046810 


Figure  B3.  Pareto  Plot  For  M2. 1  Initial  Fit 
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Actual  by  Predicted  Plot 


RSquare 

0.980716 

RSquare  Adj 

0.932507 

Root  Mean  Square  Error 

0.021131 

Mean  of  Response 

0.46052 

Observations  (or  Sum  Wgts) 

15 

Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

10 

0.09083598 

0.009084 

20.3428 

Error 

4 

0.00178610 

0.000447 

Prob  >  F 

C.  Total 

14 

0.09262208 

0.0053 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

1.0954133 

0.200116 

5.47 

0.0054 

MaxSpd 

-0.021751 

0.009478 

-2.29 

0.0834 

Accel 

-1.972906 

0.464087 

■4.25 

0.0131 

Turn  Dia 

-0.000418 

0.00018 

-2.32 

0.0814 

Max  Spd*Max  Spd 

0.0002767 

0.000132 

2.10 

0.1037 

MaxSpd*Accel 

0.0128187 

0.003735 

3.43 

0.0265 

Accel *  Accel 

-0.364444 

0.32944 

-1.11 

0.3307 

Max  Spd*Turn  Dia 

-0.000002 

9.961  e-7 

-2.30 

0.0832 

Accel *Tum  Dia 

0.0025193 

0.000522 

4.83 

0.0085 

Turn  Dia*Tum  Dia 

1.5116e-7 

5.033e-8 

3.00 

0.0398 

Turn  Dia*Tum  Dia*Accel 

-6.803e-7 

1.485e-7 

■4.58 

0.0102 

Figure  B4.  Summary  of  Model  Fit  For  M2.1  HOT 
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Error  M2.1  HOT 


Effect  Tests 
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Figure  B5.  Summary  of  Model  Fit  For  M2.1  HOT  (continued) 


Figure  B6.  Model  Fit  Error  For  M2. 1  HOT 


B.  FIT  FOR  M2.2 

The  results  for  the  NABEM  tactical  situation  2  simulations  for  the  M2. 2  MoE,  are 
feed  into  JMP.  The  fit  model  option  is  used  to  do  an  initial  fit  of  the  data  using  the 
surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 
RSquared  and  RSquared  Adjusted  values  are  0.984  and  0.953  respectively,  Figure  B4 
Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 
Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0006,  which  is  well  below  the  0.05  for  a 
desired  95%  confidence  level.  The  basic  model  statistics  look  good. 
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Second,  a  review  of  two  key  plots  is  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  B4,  all  the  data  points  fall  within  the  95%  confidence  lines  (the  dashed  red 
line),  indicating  a  good  fit  to  the  data.  Two,  the  Residual  by  Predicted  Plot,  Figure  B5, 
shows  a  fairly  evenly  shattering  of  the  points.  Some  areas  in  the  plot  have  holes,  but  that 
can  be  attributed  to  the  small  data  set  being  analyzed,  otherwise  everything  indicates  a 
good  model  fit. 

All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure  B4 
is  a  good  model  fit  to  the  data. 

For  the  model  fit  error  analysis,  Figure  B6  shows  the  resulting  distribution  for  the 
model  percent  error.  The  distribution  does  not  look  like  a  normal  distribution,  which  is 
what  would  be  expected.  In  addition,  the  mean  is  0.035,  which  is  close  to  0.0,  but  the 
standard  deviation  is  1.73,  which  is  much  higher  than  desired.  Based  on  the  model  fit 
error  analysis,  the  model  for  M2.2  does  not  perform  as  well  as  would  be  expected  from 
the  initial  analysis  of  the  fit.  This  is  due  most  likely  to  the  small  data  set  used  for  the 
analysis.  An  additional  set  of  data  points  should  be  added  to  the  analysis,  to  help  better 
define  the  model.  However,  due  to  time  and  limited  resources,  that  is  not  possible,  and 
the  fit  will  have  to  suffice  with  the  caveat  that  the  final  analysis  can  only  provide  general 
trends,  not  qualitative  answers. 
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Actual  by  Predicted  Plot 
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Figure  B7.  Summary  of  Model  Fit  For  M2. 2 
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Effect  Tests 
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Figure  B8.  Summary  of  Model  Fit  For  M2.2  (continued) 


Figure  B9.  Model  Fit  Error 
For  M2. 2 


C.  FIT  FOR  M2.3 

The  results  for  the  NABEM  tactical  situation  2  simulations  for  the  M2. 3  MoE,  are 
feed  into  JMP.  The  fit  model  option  is  used  to  do  an  initial  fit  of  the  data  using  the 
surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 
RSquared  and  RSquared  Adjusted  values  are  0.892  and  0.698  respectively,  Figure  B10 
Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 
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Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0539,  which  is  above  the  0.05  for  a 
desired  95%  confidence  level.  The  basic  model  statistics  do  not  look  great  and  deserve 
some  more  attention. 

Second,  a  review  of  two  key  plots  is  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  BIO,  all  the  data  points,  with  the  exception  of  two  points,  fall  within  the  95% 
confidence  lines  (the  dashed  red  line),  indicating  a  good  fit  to  the  data.  Two,  the 
Residual  by  Predicted  Plot,  Figure  Bll,  shows  a  fairly  evenly  shattering  of  the  points. 
Some  areas  in  the  plot  have  holes,  but  that  can  be  attributed  to  the  small  data  set  being 
analyzed,  otherwise  everything  indicates  a  good  model  fit. 

To  fix  the  fit  statistics,  higher  order  terms  are  added  to  the  model.  The  two  terms 
that  have  the  most  influence  from  the  initial  fit  are  (Turn  Dia)  and  (Accel)  (refer  to  the 
Pareto  Plot  in  Figure  B12).  Running  combinations  of  higher  order  terms  using  Turn  Dia 
and  Accel,  the  best  results  are  achieved  by  adding  the  term  [Turn  Dia  *  Turn  Dia  * 
Accel].  The  new  RSquared  and  RSquared  Adjusted  values  are  0.987  and  0.954 
respectively,  and  the  Analysis  of  Variance  the  F  statistic  was  0.0025,  Figure  B13. 

The  new  Actual  by  Predicted  Plot  (Figure  B13)  and  Residual  by  Predicted  Plot 
(Figure  Bll)  show  the  same  similar  characteristics  as  for  the  initial  fit 

All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure 
B13  was  a  good  model  fit  to  the  data. 

For  the  model  fit  error  analysis,  Figure  B15  shows  the  resulting  distribution  for 
the  model  percent  error.  The  distribution  does  not  look  like  a  normal  distribution,  which 
is  what  would  be  expected.  In  addition,  the  mean  is  0.035,  which  is  close  to  0.0,  but  the 
standard  deviation  is  1.73,  which  was  much  higher  than  desired.  Based  on  the  model  fit 
error  analysis,  the  model  for  M2.2  does  not  perform  as  well  as  would  be  expected  from 
the  initial  analysis  of  the  fit.  This  is  due  most  likely  to  the  small  data  set  used  for  the 
analysis.  An  additional  set  of  data  points  should  be  added  to  the  analysis,  to  help  better 
define  the  model.  However,  due  to  time  and  limited  resources,  that  is  not  possible,  and 
the  fit  will  have  to  suffice  with  the  caveat  that  the  final  analysis  can  only  provide  general 
trends,  not  qualitative  answers. 
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Figure  BIO.  Summary  of  Initial  Model 
Fit  For  M2. 3 


Figure  B 1 1 .  Summary  of  Initial  Model  Fit  For  M2. 3 
(continued) 


Pareto  Plot  of  Transformed  Estimates 


Term 
Max  Spd 
Accel  *Accel 
Max  Spd*Max  Spd 
Turn  Dia*Tum  Dia 
Accel 

Max  Spd*Accel 
Accel *Tum  Dia 
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Max  Spd*Turn  Dia 
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0.0094658 
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-0.0000365 


Figure  B12.  Pareto  Plot  For  M2. 3 Initial  Fit 


48 


Actual  by  Predicted  Plot 


RSquare 

0.986821 

RSquare  Adj 

0.953873 

Root  Mean  Square  Error 

0.01383 

Mean  of  Response 

0.268027 

Observations  (or  Sum  Wgts) 

15 

Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

10 

0.05728796 

0.005729 

29.9509 

Error 

4 

0.00076509 

0.000191 

Prob  >  F 

C.  Total 

14 

0.05805305 

0.0025 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

1 .6544028 

0.215907 

7.66 

0.0016 

Max  Spd 

-0.080039 

0.013049 

-6.13 

0.0036 

Accel 

-3.849062 

0.647772 

-5.94 

0.0040 

Turn  Dia 

0.0001293 

0.000059 

2.18 

0.0952 

Max  Spd*Max  Spd 

0.0010368 

0.000185 

5.60 

0.0050 

Max  Spd*Accel 

0.2072875 

0.038346 

5.41 

0.0057 

Accel *  Accel 

0.7844444 

0.215615 

3.64 

0.0220 

Max  Spd*Turn  Dia 

-6.667e-9 

6.52e-7 

-0.01 

0.9923 

Accel  *Tum  Dia 

-0.000004 

0.000033 

-0.11 

0.9159 

Turn  Dia*Tum  Dia 

-3.666e-8 

1 .533e-8 

-2.39 

0.0751 

Max  Spd*Max  Spd*Accel 

-0.00293 

0.000547 

-5.36 

0.0058 

Figure  B13.  Summary  of  Model  Fir  M2. 3  HOT 
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Figure  B14.  Summary  of  Model  Fit  For  M2. 3  HOT 
(continued) 


F igure  B 1 5 .  Model  F it 
Error  For  M2. 3  HOT 


D.  FIT  FOR  M2.4 

The  results  for  the  NABEM  tactical  situation  2  simulations  for  the  M2. 4  MoE,  are 
feed  into  JMP.  The  fit  model  option  was  used  to  do  an  initial  fit  of  the  data  using  the 
surface  response  option. 

First  is  to  determine  if  the  model  statistics  indicate  a  good  fit  to  the  data.  One,  the 
RSquared  and  RSquared  Adjusted  values  are  0.956  and  0.877  respectively,  Figure  B16 
Generally  anything  higher  than  0.80  would  be  acceptable  and  indicates  a  good  model  fit. 
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Two,  the  Analysis  of  Variance  the  F  statistic  is  0.0067,  which  is  well  below  the  0.05  for  a 
desired  95%  confidence  level.  The  basic  model  statistics  look  good. 

Second,  a  review  of  two  key  plots  is  preformed.  One,  the  Actual  by  Predicted 
Plot,  Figure  B16,  all  the  data  points,  with  the  exception  of  two  points,  fall  within  the  95% 
confidence  lines  (the  dashed  red  line),  indicating  a  good  fit  to  the  data.  Two,  the 
Residual  by  Predicted  Plot,  Figure  B17,  shows  a  fairly  evenly  shattering  of  the  points. 
Some  areas  in  the  plot  have  holes,  but  that  can  be  attributed  to  the  small  data  set  being 
analyzed,  otherwise  everything  indicates  a  good  model  fit. 

All  indications  show  that  the  model  listed  in  the  Parameter  Estimates  in  Figure 
B10  is  a  good  model  fit  to  the  data. 

For  the  model  fit  error  analysis,  Figure  B18  shows  the  resulting  distribution  for 
the  model  percent  error.  The  distribution  does  not  look  like  a  normal  distribution,  which 
is  what  would  be  expected.  In  addition,  the  mean  is  0.141,  which  is  close  to  0.0,  but  the 
standard  deviation  is  3.85,  which  was  much  higher  than  desired.  Based  on  the  model  fit 
error  analysis,  the  model  for  M2. 2  does  not  perform  as  well  as  would  be  expected  from 
the  initial  analysis  of  the  fit.  This  is  due  most  likely  to  the  small  data  set  used  for  the 
analysis.  An  additional  set  of  data  points  should  be  added  to  the  analysis,  to  help  better 
define  the  model.  However,  due  to  time  and  limited  resources,  that  is  not  possible,  and 
the  fit  will  have  to  suffice  with  the  caveat  that  the  final  analysis  can  only  provide  general 
trends,  not  qualitative  answers. 
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Profc»|t| 

Intercept 

2.8127759 

1 .293965 

2.17 

0.0818 

Max  Spd 

-0.034566 

0.075335 

-0.46 

0.6656 

Accel 

2.0120896 

2.024589 

0.99 

0.3659 

Turn  Dia 

-0.000667 

0.000722 

-0.92 

0.3976 

Max  Spd*Max  Spd 

0.0013263 

0.001047 

1.27 

0.2612 

Max  Spd*Accel 

-0.011356 

0.029692 

-0.38 

0.7178 

AcceFAccel 

-3.271806 

2.61857 

-1.25 

0.2668 

Max  SpdTurn  Da 

-5.117e-7 

0.000008 

-0.06 

0.9510 

AcceFTum  Dia 

7.5e-7 

0.000396 

0.00 

0.9986 

Turn  Dia*Turn  Dia 

1.8814e-7 

1.862e-7 

1.01 

0.3587 

Figure  B16.  Summary  of  Model  Fit  For  M2.4 
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Effect  Tests 


Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

Max  Spd 

1 

1 

0.00593924 

0.2105 

0.6656 

Accel 

1 

1 

0.02786401 

0.9877 

0.3659 

Turn  Dia 

1 

1 

0.02411475 

0.8548 

0.3976 

Max  Spd*IVbx  Spd 

1 

1 

0.04523176 

1.6033 

0.2612 

Max  Spd*Accel 

1 

1 

0.00412686 

0.1463 

0.7178 

Accel*Accel 

1 

1 

0.04404224 

1.5612 

0.2668 

Max  Spd*Turn  Da 

1 

1 

0.00011781 

0.0042 

0.9510 

Accel*Tum  Dia 

1 

1 

0.00000010 

0.0000 

0.9986 

Turn  Dia*Turn  Dia 

1 

1 

0.02879876 

1.0208 

0.3587 

Residual  by  Predicted  Plot 


0.3  —r 
0.2- 


1  aiH- 


-0.2- 


- 1 - 1 - 1 - 1 - 1 - 

2.25  2.50  2.75  3.00  3.25  3.50  3.75 

M2 .4  Predicted 


Error  M2. 4 


Quantiles 


100.0% 

maximum 

10.58 

99.5% 

10.58 

97.5% 

10.58 

90.0% 

5.88 

75.0% 

quartile 

1.83 

50.0% 

median 

0.39 

25.0% 

quartile 

-2.18 

10.0% 

-4.66 

2.5% 

-6.90 

0.5% 

-6.90 

0.0% 

minimum 

-6.90 

Moments 

Mean 

0.1400942 

Std  Dev 

3.8536282 

Std  Err  Mean 

0.9950025 

upper  95%  Mean 

2.2741623 

lower  95%  Mean 

-1.993974 

N 

15 

Figure  B17.  Summary  of  Model  Fit  For  M2.4  (continued) 


Figure  B18.  Model  Fit 


Error  For  M2.4 
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